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ABSTRACT 


The  purpose  of  this  project  is  to  provide  a  final  business  case  analysis  of  the 
technologies  demonstrated  in  the  Agile  Rapid  Global  Combat  Support  (ARGCS)  system. 
Three  theses  and  one  professional  paper  have  previously  analyzed  the  ARGCS  Advanced 
Concept  Technology  Demonstration  (ACTD)  return  on  (proposed)  investment.  This 
project  will  review  those  works,  provide  a  history  of  the  ARGCS  ACTD,  discuss  the 
goals  for  the  ACTD  and  compare  alternatives.  In  particular,  this  analysis  builds  on 
previous  efforts,  including  the  Discrete-Event  Simulation  Model  developed  in  the  Bello, 
Rios,  Carpenter  thesis  (December  2006). 

The  ultimate  goal  of  this  project  is  to  assist  in  the  analysis  of  the  ARGCS 
technologies  and  what  benefit  they  would  provide  if  included  in  the  proposed  next 
generation  of  Naval  Aviation  test  equipment,  currently  called  the  Enhanced  Consolidated 
Automated  Support  System  (ECASS). 
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I.  INTRODUCTION 


A.  PURPOSE 

The  purpose  of  this  project  is  to  provide  a  final  business  case  analysis  of  the 
technologies  demonstrated  in  the  Agile  Rapid  Global  Combat  Support  (ARGCS)  system. 
Three  theses  (1.  Crosby,  David,  “Business  Case  Analysis:  Agile  Rapid  Global  Combat 
Support.  2.  Jupiter,  David  L.,  Reuter,  Lisa  J.,  “The  Development  of  a  Business  Case 
Analysis  for  the  acquisition  of  the  Agile  Rapid  Global  Combat  Support  System  used  for 
the  United  States  Marine  Corps’  Ground  Equipment.”  3.  Bello,  Brian,  “Analysis  of 
Agile  Rapid  Global  Combat  Support  System  Technologies  in  the  support  of  the  F/A- 
18C”)  and  one  professional  paper  (Franck,  Raymond,  “Agile  Rapid  Global  Combat 
Support,  Business  Case  Analysis,  Version  1.0)  have  previously  analyzed  the  ARGCS 
Advanced  Concept  Technology  Demonstration  (ACTD)  return  on  (proposed)  investment. 
This  project  will  review  those  works,  provide  a  history  of  the  ARGCS  ACTD,  discuss  the 
technology  developed  in  the  ARGCS  project,  the  goals  for  the  ACTD  and  compare 
alternatives.  In  particular  this  analysis  builds  on  previous  efforts,  including  the  Discrete- 
Event  Simulation  Model  developed  in  the  Bello,  Rios,  Carpenter  thesis  (December  2006). 

The  purpose  of  this  project  is  to  assist  the  Department  of  Defense  in  its  analysis  of 
ARGCS  technologies  and  what  benefit  they  would  provide  if  included  in  the  next 
generation  of  Naval  Aviation  test  equipment,  currently  called  the  Enhanced  Consolidated 
Automated  Support  System  (ECASS). 

B.  RESEARCH  QUESTIONS 

This  ARGCS  analysis  will  focus  on  identifying  benefits  associated  with  the 
technologies  included  in  the  ACTD.  The  following  questions  will  be  the  concentration  of 
this  thesis: 


1 


1.  Primary  Questions 

1 .  What  is  ARGCS?  What  capability  gaps  does  ARGCS  address? 

2.  What  are  the  ARGCS  ACTDs,  how  do  they  interface  with  CASS  and 
ECASS? 

3.  What  is  a  BC  A?  How  will  this  address  the  ROI  of  ARCS? 

4.  What  is  the  best  estimate  of  the  ARGCS  ROI  based  on  the  notional 
ECASS  system? 

2.  Secondary  Questions 

1 .  What  is  the  ARGCS  program  history  and  structure? 

2.  What  is  the  nature  of  the  test  program? 

3.  What  are  the  ARGCS  prototypes  and  to  what  specifications  were  they 
built? 

4.  What  are  the  resource  impact  areas  (costs/savings)?  To  what  extent  can 
they  be  quantified? 


C.  BACKGROUND 

In  the  mid  1980s,  the  U.S.  Navy  became  the  first  Service  to  consolidate  current 
Automated  Test  Systems  (ATS)  into  a  common-consolidated  system  of  Automated  Test 
Equipment  (ATE)  with  a  view  toward  reducing  support  costs.  The  first  result  of  those 
efforts  was  the  Consolidated  Automatic  Support  System  (CASS).  Considered  the  largest 
automated  test  support  program  and  the  first  recognized  U.S  Department  of  Defense 
(DoD)  standard  test  system,  CASS  provides  general-purpose  analog  and  digital  test 
capabilities.  In  2004  Naval  Air  Systems  Command’s  (NAVAIR)  Program  Management 
Activity  (PMA)  office  received  permission  from  The  Joint  Requirements  Oversight 
Council  (JROC)  to  begin  the  CASS  modernization  program,  the  titled  the  enhanced 
Consolidated  Automatic  Support  System  (ECASS). 

In  2005,  the  Naval  Postgraduate  School  (NPS)  was  approached  by  NAVAIR  and 
OSD  to  participate  in  the  analysis  of  the  ARGCS  technologies.  The  first  project 
established  by  NPS  was  a  thesis  authored  by  CDR  David  Crosby  in  June  2006.  The 
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thesis  entitled  “Development  of  a  Business  Case  Analysis  for  the  Acquisition  of  the  Agile 
Rapid  Global  Combat  Support  System,”  focused  on  the  economic  potential  of  the 
ARGCS  system.  This  project  was  initiated  with  the  goal  to  build  a  business  case  model 
to  assess  the  potential  savings  if  the  next  generation  of  Naval  Aviation  test  equipment 
were  augmented  with  the  ARGCS  system  and  produce  an  actual  cost/benefit  analysis. 
The  benefits  to  be  evaluated  in  this  thesis  included  cost  savings,  efficiency 
improvements,  as  well  as  non-monetary  readiness  benefits  obtained  through  better  failure 
rates  prediction  and  maintenance  diagnostics.  In  the  absence  of  test  data,  a  framework 
baseline  analysis  was  conducted  using  tools  such  as  a  Return  on  Investment  (ROI), 
establishment  of  expected  Net  Present  Value  (NPV)  and  Internal  Rate  of  Return  (IRR), 
Manpower  Sustainment  calculations  and  a  Sensitivity  Analysis.  As  a  result  of  ARGCS 
testing  being  delayed,  less  than  25  percent  of  the  costs  associated  with  ARGCS  were 
known  to  any  degree  of  certainty.  As  a  result  the  project  was  modified  to  produce  a 
model  for  application  when  further  data  became  available.  The  results  of  the  project 
recommended  that  a  revised  Business  Case  Analysis  (BCA)  be  conducted  at  the 
conclusion  of  the  ACTD  testing. 

The  second  project  conducted  by  NPS  in  support  of  the  ARGCS  evaluation  was  a 
thesis  by  Lisa  Jupiter  and  David  Reuter,  “The  Development  of  a  Business  Case  Analysis 
for  the  Acquisition  of  the  Agile  Rapid  Global  Combat  Support  System  to  be  used  for 
Marine  Corps  Ground  Equipment  (2006).”  This  project  analyzed  ARGCS  technologies 
and  used  BCA  methodology  to  detennine  the  advisability  for  the  USMC  to  incorporate 
ARGCS  technologies  in  the  proposed  ATS  replacement  instead  of  the  Third  Generation 
Test  System  (TETS)  that  was  introduced  to  the  field  in  FY-2000.  This  thesis  was  also 
limited  by  lack  of  data.  Nevertheless,  this  thesis  provided  a  useful  assessment  of  second- 
generation  USMC  test  equipment  and  an  excellent  assessment  of  the  virtues  of  the  new 
TETS.  It  also  provided  the  necessary  groundwork  for  comparing  ARGCS  with  TETS  in 
increased  operational  availability  of  land  combat  weapons  systems. 

The  third  project  conducted  by  NPS  in  support  of  the  ARGCS  evaluation  was  a 
thesis  authored  in  December  of  2006  by  Alfred  P.  Bello  III,  Frankie  Rios  and  Robert  B. 
Carpenter,  “Discrete -Event  Simulation  Modeling  of  the  Repairable  Inventory  Process  to 
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Enhance  the  ARGCS  Business  Case  Analysis.”  This  project  used  a  simulation  model, 
with  pre-existing  maintenance  and  supply  data  to  more  accurately  estimate  maintenance 
and/or  supply  cost  benefits  from  fielding  ARGCS  technologies.  The  results  of  the  initial 
run  of  the  simulation  model  indicated  an  increase  in  operational  availability  as  well  as 
cost  savings. 

The  latest  project  completed  at  NPS  assessing  the  value  of  the  ARGCS  system 
was  an  interim  Business  Case  Analysis  authored  by  Professor  Raymond  Franck  and 
Carmelita  Troy  in  January  2007.  Revised  in  May  of  2007,  this  interim  BCA,  based  on 
the  data  then  available,  offered  an  assessment  of  the  Return  on  Investment  from  the 
ARGCS  ACTD  and  recommended  a  more  definitive  BCA  at  the  conclusion  of  the 
ARGCS  tests.  This  project  provided  an  excellent  framework  for  a  thorough  analysis  of 
the  ROI  of  ARGCS,  cost  savings  related  to  the  potential  reduction  in  False  Pulls 
associated  with  the  technologies  and  how  this  reduction  can  increase  Operational 
Availability  of  the  F/A-18C  and  potentially  reduce  the  number  of  aircraft  required  in  the 
fleet. 

Each  of  these  projects  enhances  the  fonnulation  for  a  final  BCA  to  include  an 
updated  Discrete-Event  simulation  using  data  obtained  from  the  Limited  Military  Utility 
Assessment  (LMUA)  at  Naval  Air  Station  (NAS)  Lemoore  in  August  of  2007. 

D.  BACKGROUND  ON  THE  FUNDING  FOR  THE  ARGCS  ACTD 

As  referenced  earlier,  the  Navy  received  permission  from  the  JROC  in  FY  2004  to 
develop  a  replacement  ATS  for  CASS,  the  replacement  system,  ECASS  is  set  to  start 
Limited  Rate  Initial  Production  (LRIP)  in  FY  2012.  This  system  targets  updating 
relatively  old  test  equipment,  and  upgrading  test  capabilities  to  support  future  weapons 
systems.”1  In  line  with  this  initiative,  the  DoD’s  Automatic  Test  Systems  Management 
Board  “launched  ARGCS  to  bring  about  the  development  of  a  single  tester  that  takes 


1  Courtney  E.  Howard,  Military  &  Aerospace  Electronics,  “Attention  turns  to  RF  and  microwave  test 
and  measurement,”  May  2007. 
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advantage  of  an  open-architecture  system  of  modular  hardware  and  flexible  software  to 
test  and  maintain  combat  systems  across  all  branches  of  the  U.S.  Military.”2 


OSD  RTD&E  Project  P648  announced  the  ARGCS  project,  with  the  following  guidance: 

The  Joint  Requirements  Oversight  Council  (JROC)  validated  the 
capability  need  for  ARGCS  as  an  FY-04  start  which  is  using  advanced 
technologies  to  demonstrate  a  family  of  testers  for  electronic  components 
and  provide  unprecedented  interoperability  between  weapon  systems, 
Services,  and  levels  of  maintenance.  This  will  reduce  costs  and  the 
proliferation  of  testers  while  improving  the  availability  and  performance 
of  weapons  systems.  In  addition,  ARGCS  will  demonstrate  technologies 
to  facilitate  net-centric  diagnostics  by  capturing  historical  logistics  data 
and  developing  an  expert  support  system  that  will  further  reduce  repair 
times  and  costs,  as  well  as  future  sparing  requirements.  Outputs  and 
efficiencies  will  include  increases  in  perfonnance  and  test  accuracy, 
interoperability  between  Services,  reduced  logistics  and  weapons  system 
support  costs,  and  reduced  proliferation  of  automatic  test  systems  in  the 
future.  (100%  interoperability,  Time  to  field  -  on  year  or  less,  40% 
reduced  time  to  diagnose  and  repair,  proliferation  of  systems  -  reduced 
footprint  by  50%,  Scalability  of  systems  -  100%).  The  ARGCS 
technology  will  be  transitioned  to  the  Services  through  existing  automated 
test  programs  of  record.  The  user  sponsor  is  the  U.S.  Pacific  Command 
and  the  lead  service  is  the  Navy. 


FY  2008  planned  output:  Complete  the  ACTD  and  continue  to  evaluate 
ARGCS  military  utility  through  the  EUE.  Support  transitions  of  ARGCS 
technologies  and  products  into  service  programs  of  record.  3 


Also  from  a  recent  United  States  Pacific  Command  (USPACOM)  “White  Paper”: 

Utilizing  existing  Services’  infrastructure  and  interfacing  to  the  Services’ 
current  logistic  systems,  ARGCS  will  provide  the  warfighter  the  ability  to 
access  SMEs  to  assist  in  maintenance  actions  in  the  field.  It  also  provides 
the  infrastructure  to  link  all  major  data  repositories,  which  provides  all 
warfighters  with  the  most  up-to-date  logistics,  historical  and  training 
information  in  order  to  ensure  the  shortest  test  times  and  the  most  accurate 
fault  diagnosis.  Net-centric  diagnostics  electronically  integrates  test 

2  Howard,  Military  &  Aerospace  Electronics. 

3  OSD  RDT&E  Justification  (R2a  Exibit),  Joint  Capability  Technology  Demonstrations,  Project  P648, 
February  2007. 
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information  across  all  test  levels  and  at  all  locations,  providing  flexible 
interoperability.  Historical  maintenance  and  logistics  data  is  stored  and 
made  available  to  users  at  all  levels  of  maintenance,  providing  global 
connectivity.2 3 4 

The  industry  focus  for  ARGCS  and  ECASS  is  inline  with  the  restructuring 
initiatives  of  the  Defense  Advanced  Research  Projects  Agency  (DARPA)  and  the 
acquisition  guidelines  set  forth  in  the  DoD  Joint  Capabilities  Integration  Development 
System  (JCIDS)  instruction  5000.2.  The  Navy  has  traditionally  sought  bids  from  a 
number  of  contractors,  in  the  case  of  CASS,  “Boeing  won  the  right  to  develop  CASS 
after  competing  against  Grumman  and  Lockheed  in  1987,”  5  In  the  case  of  ARGCS,  in 
September  of  2004,  Northrop  Grumman  was  awarded  a  $26. 7M  contract  to  develop  the 
ARGCS  ACTD  prototypes.6 

E.  SCOPE  AND  METHODOLOGY 

The  scope  of  this  project  will  include:  (1)  a  background  on  the  ARGCS  system, 
(2)  a  methodology  of  the  research  and  a  review  of  the  process  used  in  a  BCA,  (3)  an 
analysis  (methodology  and  content)  of  the  LMUA  and  JMUA  testing  procedures  data, 
and  (4)  A  final  BCA  using  an  analysis  of  alternatives  using  existing  cost  information 
associated  with  CASS  and  an  updated  simulation  using  the  model  developed  in  the  Bello 
et  al.,  project. 


The  methodology  used  in  this  thesis  research  will  consist  of  the  following  steps. 

1 .  Conduct  a  literature  search  for  background  on  the  technologies  used  in  the 
development  of  ARGCS  in  magazine  articles,  JITC  discussions,  NPS  theses  and 
other  library  information  resources. 

2.  Conduct  a  comparison  of  BCA  methodology  and  requirements,  between  that 
used  by  the  Defense  Acquisition  University  (DAU)  and  that  preferred  by  civilian 
enterprise. 

3.  Conduct  a  review  of  the  current  ARGCS 


4  White  Paper. 

5  CASS  website:  www.acq.osd.mil/ats/cass.htm 

6  Northrop  Grumman  News  Release,  September  2004 
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4.  A  discussion  of  emergent  technologies  with  the  ARGCS  ACTD. 

5.  A  review  of  the  current  CASS  equipment  and  an  explanation  of  how  the  list  of 
expectations  for  the  ARGCS  technologies  was  generated. 

6.  A  review  of  the  JROC  requirements  for  the  ARGCS  testing  and  development. 

7.  A  summary  of  major  cost  assumptions  associated  with  the  proposed 
development  of  EC  ASS. 

8.  An  analysis  of  the  potential  for  reduced  “false  pulls”  and  increased  operational 
availability  of  the  F/A-18  C  due  to  the  implementation  of  the  ARGCS 
technologies. 

9.  An  analysis  of  the  two  existing  alternatives;  keep  CASS  in-service  until  all 
technologies  included  in  ARGCS  have  been  fully  evaluated  or  proceed  with 
ECASS  incorporating  the  proposed  ARGCS  technologies. 
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II.  DISCUSSION  OF  EXISTING  TECHNOLOGIES  USED  IN  THE 

ARGCS  SYSTEM 


A.  ARGCS  TECHNOLOGIES 

Due  to  aging  systems  and  rising  O&M  costs,  new  technologies  have  been 
researched  and  included  in  the  ARGCS  prototypes.  The  goal  is  to  demonstrate  several 
functional  capabilities  that  will  enhance  Navy  aircraft  maintenance  operations.7  Toward 
that  effort  ARGCS  incorporates  very  modern,  but  proven,  capabilities  to  improve  test 
analysis  over  the  existing  system;  the  most  significant  are  Open  Systems  Architecture,  a 
Dual-Core  processor  and  Synthetic  instrumentation.  The  ARGCS  ACTD  demonstrated 
an  open-architecture  and  upgradeable  Radio  Frequency  (RF)  test  subsystem  based  on 
Synthetic  Instruments.8 9 

This  portion  of  the  research  will  lay  out  a  basic  explanation  of  these  technologies. 


B.  OPEN  SYSTEM  ARCHITECTURE 

Open  systems  architecture  is  a  standard  that  describes  the  layered  hierarchical 
structure  of  the  systems  overall  design,  which  provides  the  following91 

•  Enables  system  design,  development,  installation,  operation, 
improvement,  and  maintenance  to  be  performed  at  a  given  layer  or  layers 
in  the  hierarchical  structure 

•  Allows  each  layer  to  provide  a  set  of  accessible  functions  that  can  be 
controlled  and  used  by  the  functions  in  the  layer  above  it 

•  Enables  each  layer  to  be  implemented  without  affecting  the 
implementation  of  other  layers 

•  Allows  the  alteration  of  system  performance  by  the  modification  of  one  or 
more  layers  without  altering  the  existing  equipment,  procedures,  and 
protocols  at  the  remaining  layers 


7“ Agile  Rapid  Global  Combat  Support  Advanced  Concept  Technology  Demonstration  Integrated 
Assessment  Plan”  Joint  Interoperability  Test  Command,  FT  Huachuca,  AZ,  May  2006. 

8ARGCS  ACTD  White  Paper. 

9Camegie  Mellon  University,  Software  Engineering  Institute  Website. 
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C.  DUAL  CORE  PROCESSORS 

A  dual-core  Central  Processing  Unit  (CPU)  combines  two  independent  cores  into 
a  single  package  composed  of  a  single  integrated  circuit.10 

The  proximity  of  dual  multiple  CPU  cores  on  the  same  chip  allows  the  circuitry  to 
operate  at  a  much  higher  clock  rate  than  is  possible  if  the  signals  have  to  travel  off-chip. 
Combining  equivalent  CPUs  on  a  single  chip  significantly  improves  the  perfonnance  of 
operations.  Put  simply,  this  means  that  signals  between  different  CPUs  travel  shorter 
distances,  and  therefore  those  signals  degrade  less.  These  higher  quality  signals  allow 
more  data  to  be  sent  in  a  given  time  period  since  individual  signals  can  be  shorter  and  do 
not  need  to  be  repeated  as  often  11 

Assuming  that  the  chip  can  fit  into  the  package,  physically,  the  multi-core  CPU 
designs  require  much  less  Printed  Circuit  Board  (PCB)  space  than  multi-chip  designs. 
Also,  a  dual-core  processor  uses  slightly  less  power  than  two  coupled  single-core 
processors,  principally  because  of  the  increased  power  required  to  drive  signals  external 
to  the  chip  and  because  the  smaller  silicon  process  geometry  allows  the  cores  to  operate 
at  lower  voltages.12 

Simply  stated  the  improved  capability  of  the  CPU  in  ARGCS  will  allow  much 
faster  TPS  times  as  the  TPS  technology  improves  to  match  up  with  the  newer  CPU 
technology. 

D.  SYNTHETIC  INSTRUMENTATION 

The  term  “synthetic  instrumentation  (SI)”  was  coined  by  the  U.S.  Department  of 
Defense’s  Next  Generation  Automatic  Test  Systems  (NxTest)  Integrated  Product  Team 
(IPT)  to  describe  a  new  test  architecture  that  would  support  the  charter  of  its  group.  In 
April  2002,  the  DoD  Automatic  Test  Systems  (ATS)  established  the  NxTest  IPT  with  two 


10Icronic  Website. 

11 Ibid. 

12  RFDesign  Website. 
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main  goals:  to  reduce  the  total  acquisition  and  support  costs  of  DoD  ATS  and  to  improve 
the  inter-operability  of  the  armed  services  ATS  functions.13 

This  IPT  established  a  number  of  initiatives,  including  the  idea  of  an  architectural 
approach  to  test  system  design  and  implementation  that  is  now  referred  to  as  synthetic 
instrumentation.  This  architecture  is  part  of  an  overall  open  architecture  system  that  can 
support  new  test  needs  and  pennit  flexible  updates  and  the  addition  of  new  technology 
with  minimal  impact  on  existing  ATS  components.14 

The  IPT  created  a  focus  group  known  as  the  Synthetic  Instrument  Working  Group 
(SIWG);  the  SIWG  has  defined  synthetic  test  systems  as  follows: 

A  recon  figurablc  system  that  links  a  series  of  elemental  hardware  and 

software  components,  with  standardized  interfaces,  to  generate  signals  or 

make  measurements  using  numeric  processing  techniques.15 

The  goal  for  ARGCS  is  to  use  this  type  of  technology  to  support  both  simple  and 
complex  system  configurations.  The  flexibility  it  provides  will  allow  the  U.S.  Air 
Force,  Army,  Marine  Corps,  and  Navy  to  individually  procure  systems  configured  for 
their  specific  operational  scenarios.16 

This  SI  software,  is  what  enables  the  use  of  legacy  TPS  without  modification. 
This  software  can  translate  “traditional  instrument  programming  statements”i7  to 
equivalent  test  and  measurement  functions  of  the  synthetic  system.  Demonstrating  the 
ability  to  preserve  TPS  software  investment  more  efficiently  than  previous  generations  of 
equipment  is  one  of  the  major  payoffs  expected  of  synthetic  test  solutions.18 


13  RFDesign  Website. 

14  Ibid. 

15  Ibid. 

16  Ibid. 

17  Ibid. 

18  Ibid. 
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III.  BCA  METHODOLOGY 


A.  COMPARISON  OF  APPROACHES 

A  business  case  analysis,  often  referred  to  as  a  “Business  Plan,”  when  prepared 
for  private  enterprise,  is  meant  to  assess  the  business  rationale  for  choosing  or  not 
choosing  a  particular  project  and  to  clarify  the  parameters  and  management  factors 
involved  in  the  decision.  It  is  one  tool  used  to  evaluate  projects. 

A  BCA  in  private  enterprise  is  usually  an  encompassing  discussion  of  the 
business  reasons  used  to  analyze  choices.  According  to  Demand  Econometrics,  “The 
business  case  analyzes  the  current  situation,  metrics  to  indicate  what's  wrong  or  what 
needs  to  be  improved,  and  the  underlying  reasons  why.”19  The  BCA  usually  includes 
relevant  background  infonnation  about  the  problem,  as  well  as  a  discussion  of  history, 
competitive  strategy,  risk  factors,  and  external  market  considerations.  The  purpose  is  to 
provide  a  strategic  look  at  elements  that  are  being  considered  in  making  a  change  in  the 
direction  of  the  business.  An  analysis  of  risk  is  generally  a  significant  feature.  A 
thorough  review  of  the  risks  of  both  the  status  quo  and  the  risks  of  change  is  used  to 
provide  a  more  comprehensive  discussion  of  the  reasons  that  a  proposed  action  is  being 
considered. 

In  the  case  of  OSD,  BCAs  are  used  as  a  tool  to  support  acquisition  decisions.  The 
BCA  provides  the  acquiring  authority  an  analysis  of  quantifiable  factors,  including  costs, 
and  non-quantifiable  factors  such  as  the  potential  joint  use.  These  BCAs  tend  to  include 
performance,  reducibility,  reliability,  maintainability,  and  supportability  calculations. 

A  more  appropriate  comparison  of  the  OSD  BCA  to  a  similar  civilian  tool  would 
be  a  comparison  with  a  private  enterprise  Cost  Benefit  Analysis  (CBA).  OSD  BCAs  are 
generally  much  less  concerned  with  assessment  of  strategies  than  the  BCAs  in  the 
commercial  sector.  As  referenced  earlier,  OSD  uses  the  framework  they  have  established 


19  Demand  Econometrics  Website. 
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for  a  BCA  to  assist  in  making  single  program  acquisition  decisions,  not  strategic 
decisions  as  to  the  direction  of  military  forces. 

A  commercial  CBA  compares  what  a  new  item  will  cost  with  its  expected 
benefits  to  the  business.  Much  like  the  OSD  BCA,  a  commercial  CBA  is  a  decision  tool 
used  to  help  answer  "should  we"  and  "what  if'  questions,  such  as,  "should  we  fund  this 
project?"  and  "what  happens  if  we  do?"  A  CBA  uses  tools,  like  an  ROI  analysis,  to 
compare  the  value  of  investments,  particularly,  competing  projects.20 

Both  the  OSD  BCA  and  commercial  CBA  are  basically  an  attempt  to 
mathematically  quantify  the  benefits  of  a  particular  decision.  They  both  attempt  to 
simulate  (model)  the  expected  change  brought  about  by  the  purchase  or  acquisition  of  a 
particular  program  or  item. 

The  origin  of  BCA/CBAs  actually  lies  within  the  military,  in  the  1920s,  the  US 
Army  Corps  of  Engineers  developed  a  process  for  economically  evaluating  which  public 
projects  to  pursue.  Economists  formulated  the  Anny's  process,  and  have  been  formalizing 
and  standardizing  CBA  methodology  since  the  1950s.21 

B.  OSD  BCA  REQUIRMENTS 

The  basic  requirements  of  defense  acquisition  focused  BCA  include: 

1.  An  introduction  and  definition  of  the  system  being  analyzed,  its  purpose 
and  why  it  is  necessary.  It  should  also  present  the  objectives  of  the 
system. 

2.  The  methodology  and  assumptions  being  used  during  the  analysis 
including  the  rationale  used  in  defining  the  scope  of  the  analysis.  A 
description  of  the  costs  and  expected  benefits  over  a  specified  timeframe 
must  be  included. 

3.  The  organizational  impacts  both  financial  and  non-financial  associated 
with  the  scenarios  being  compared. 

4.  If  possible  a  risk  assessment  should  be  conducted  within  the  analysis  of 
alternatives. 


20  Demand  Econometrics  Website. 

21  Ibid. 
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5.  Conclusions  and  recommendations  for  specific  actions  based  on  the  initial 
objectives  of  the  system  being  considered 

6.  The  final  BCA  documents  assess  the  alternatives  considered,  factors 
involved,  both  tangible  and  intangible  and  the  proposed  benefits  of  the 
system  and  a  quantification  of  the  return  on  investment.  22 


C.  SCOPE  OF  THIS  BCA 

Using  the  guidelines  set  forth  by  the  DAU,  this  BCA  will  meet  OSD  needs 
through  a  comparison  of  alternatives.  In  this  case,  the  options  are  (1)  delaying  the 
development  of  ECASS  (until  ARGCS  can  more  thoroughly  be  evaluated)  while 
extending  the  life  of  the  existing  CASS  ATE  and  (2)  developing  ECASS  using  the 
ARGCS  technologies  and  taking  the  risk  that  the  existing  emergent  technologies 
(especially  network-enabled  maintenance)  within  the  system  will  develop  and  become  a 
viable  part  of  the  new  equipment.  The  following  is  an  outline  of  the  analysis: 


1.  A  review  of  the  ARGCS  system  and  the  objectives  associated  with  its 
development. 

2.  A  description  of  the  existing  alternatives.  To  be  included  are  the 
contingencies  associated  with  each  option.  The  focus  is  on  dollar  costs 
associated  with  existing  inventory  and  sustainment  costs  versus 
investment  costs  and  expected  savings  associated  with  the  new 
technologies. 

3.  The  methodology  and  assumptions  being  used  during  the  analysis 
including  the  rationale  used  in  defining  the  scope  of  the  analysis. 

4.  An  assessment  of  the  expected  risks  and  consequences  associated  with 
each  alternative. 

5.  A  presentation  of  the  data  from  the  basic  qualitative  analysis  and  the 
quantitative  data  obtained  from  the  updated  Discrete -Event  Simulation 
Model 

6.  A  discussion  of  the  conclusions  and  recommendations  based  on  the  initial 
objectives  of  the  ARGCS  system  and  the  requirements  for  ECASS. 


22  Defense  Acquisition  University  Website. 
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IV.  ARGCS  HISTORY  AND  TESTING  PROTOCOL 


A.  BACKGROUND 

The  goal  of  the  ARGCS  ACTD  is  to  use  emerging  test  technologies  to  integrate 
automatic  test  system  (ATS)  hardware  and  software  with  a  net-centric  support  system  to 
improve  electronic  systems  maintenance.  The  goal  is  for  ARGCS  to  provide  diagnostic 
support  at  the  organizational,  intennediate,  depot  and  factory  levels  of  maintenance. 
ARGCS  is  intended  to  be  easily  and  quickly  deployable  worldwide  with  reduced  airlift 
and  logistics  footprint  requirements  over  CASS.23 

B.  DEMONSTRATION  STRATEGY 

The  ARGCS  ACTD  demonstration  program  has  included  a  Limited  Military 
Utility  Assessment  (LMUA)  conducted  with  the  integrating  contractor’s  (Northrop 
Grumman)  facility  and  three  Joint  Military  Utility  Assessments  (JMUAs)  conducted  at 
Service  maintenance  facilities.  The  JROC  selected  United  States  Pacific  Command 
(USPACOM)  as  the  ACTD  sponsor,  and  then  they  selected  the  Joint  Interoperability  Test 
Command  (JITC)  to  conduct  these  assessments.24 

The  LMUA  was  intended  to  demonstrate  ARGCS  capability  to  provide  diagnostic 
support  for  all  services;  to  share  test  and  repair  data  from  all  levels  of  maintenance;  and 
to  collaborate  with  other  maintenance  personnel  or  SMEs.  The  JMUA  focused 
specifically  on  the  individual  Services  and  potential  ARGCS  integration  with  proposed 
next  generation  ATS. 

C.  ASSESSMENT  STRUCTURE 

Because  ARGCS  is  not  an  acquisition  program  of  record,  there  is  not  a  formal 
requirements  document.  Therefore,  USPACOM  (OSD’s  designated  agent)  developed  the 

23  “Agile  Rapid  Global  Combat  Support  System  Advanced  Concept  Technology  Demonstration 
Integrated  Assessment  Plan”  Joint  Interoperability  Test  Command,  FT  Huachuca,  AZ,  May  2006. 

24  Ibid. 
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“ARGCS  Management  Plan”  to  determine  the  assessment  requirements.  This  plan  is  the 
main  focus  of  the  ARGCS  IAP  and  delineates  the  controls  and  measures  to  be  used  the 
assessments  of  the  technologies.  The  plan’s  statement  of  need  lists  six  operational 
problems  that  confront  the  Services  in  the  area  of  weapons  systems  support  Joint 
Warfighter  Problems  (JWP)  in  the  Concept  of  Operations  (CONOPS)  developed  by  the 
JROC. 


Table  1.  The  Joint  Warfighter  Problems  identified.25 


JWP 

No. 

Weapons  Systems  Support  Problem 

1 

Lack  of  interoperability  between  Services'  support  equipment 

2 

Support  equipment  not  available  for  newly  fielded  weapons  systems 

3 

Support  equipment  upgrades  too  slow  for  weapons  systems  support 

4 

High  support  costs 

5 

Huge  logistics  footprint 

6 

High  false  failure  rates 

The  next  step  was  to  identify  the  Critical  Operational  Issues  (COI)  that  relates 
assessment  of  ARGCS  to  the  JWPs.  In  order  to  resolve  the  COIs,  JITC  assigned 
Measures  of  Performance  (MOP)  and  suitability  (MOS)  to  identify  areas  within  each  COI 
for  assessment.26 

To  better  understand  MOPs,  start  with  the  overall  objective  for  the  ARGCS 
testing,  to  determine  if  the  technologies  have  military  usability.  Each  Service  already  has 
an  ATS,  so  in  order  for  ARGCS  to  have  military  utility  it  must  improve  the  status  quo. 
For  example,  if  a  metric  for  improvement  is  doing  the  same  job  faster,  a  MOP  is  made 
that  relates  to  time.27  Specific  examples  of  this  follow  in  the  section  below. 


25  “Agile  Rapid  Global  Combat  Support  System  Advanced  Concept  Technology  Demonstration 
Integrated  Assessment  Plan”  Joint  Interoperability  Test  Command,  FT  Huachuca,  AZ,  May  2006 

26  Ibid. 

27Anthony  Mason,  JITC  ARGCS  Program  Manager,  Email  June  5,  2007. 
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MOPs  are  the  evaluation  controls  for  the  ARGCS  assessment  program.  MOPs 
establish  the  metric  (time),  what  will  actually  be  measured  (diagnostic  time),  and  how 
well  the  system  should  perform  (i.e.,  a  threshold;  such  as,  15  percent  less  than  the  current 
standard).28 

Other  data  being  gathered  during  the  assessments  will  be  interviews  of  individual 
maintainers  to  identify  possible  impacts  relating  to  Doctrine,  Organization,  Training, 
Material,  Leadership,  People  and  Facilities  (DOTMLPF). 

During  each  demonstration,  maintainers  are  interviewed  with  a  focus  on  the 
following  areas: 

•  Doctrine:  procedures  for  transfonning  new  technologies  into  joint 

capabilities 

•  Organization:  how  ARGCS  technologies  enhance  mission  responsiveness 

•  Training:  skills  required  to  improve  individual  and  unit  effectiveness 

•  Material:  equipment  needed  to  support  effective  use  of  the  ARGCS 
technologies 

•  Leadership:  whether  ARGCS  technologies  assist  in  clearing  the  inevitable 
fog  of  war  related  to  equipment  maintenance  status 

•  People:  appropriate  personnel  required  to  operate  the  ARGCS 

technologies  and  their  locations 

•  Facilities:  whether  ARGCS  technologies  are  capable  of  operating  under 
the  current  communications  architecture.  29 

The  JITC  analysts  will  validate  the  DOTMLPF  impacts  against  the  ARGCS  Concept  of 
Operations.  DOTMLPF  impacts  will  be  updated  throughout  the  course  of  the  ACTD  and 
will  be  incorporated  into  the  comprehensive  JMUA  report  at  the  end  of  the  ARGCS 
ACTD. 

D.  METHODOLOGY 

The  JITC  has  addressed  ARGCS  the  COIs,  MOSs,  and  MOPs  by  collecting  data 
in  the  context  of  realistic  scenarios  and  real-world  missions.  To  assess  the  utility  of  the 

28  Anthony  Mason,  JITC  ARGCS  Program  Manager,  Email  June  5,  2007. 

29  “Agile  Rapid  Global  Combat  Support  Advanced  Concept  Technology  Demonstration  Integrated 
Assessment  Plan”  Joint  Interoperability  Test  Command,  FT  Huachuca,  AZ,  May  2006. 
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physical  and  perfonnance  characteristics  of  the  ARGCS  Automatic  Test  System  (ATS)  it 
compared  to  the  ATSs  currently  employed  by  the  Services.  Maintainers  first  diagnosed 
real  problems  on  their  service’s  legacy  test  equipment  using  the  selected  production 
TPSs.  The  maintainers,  the  TPSs,  the  faulty  unit(s)/component(s),  and  a  fully  operational 
component  were  then  transported  to  the  ARGCS  ATS  location  for  diagnoses. 

The  JITC  then  developed  a  functionality  and  usability  survey  and  administered  it 
at  the  end  of  each  demonstration  to  assess  maintainer  opinions  of  various  aspects  of  each 
COI.  Each  question  had  a  space  for  maintainers  to  provide  additional  comments, 
explanations,  or  recommendations.  The  JITC  then  validated  the  survey  responses  by 
interviewing  respondents  before  compiling  and  reporting  survey  results.30 

E.  PERSONNEL  CONTROLS 

In  an  effort  to  gauge  the  usability  of  ARGCS,  average  to  above  average 
maintainers  were  requested  at  NAS  Lemoore  and  other  sites.  Other  than  competency  on 
the  CASS,  the  only  other  criterion  for  selection  as  a  test  participant  is  if  the  host  feels  a 
particular  individual  has  a  voiced  interest  in  the  ACTD  and  wishes  to  participate.31 

In  addition  to  participating  in  the  physical  assessment  of  the  ACTD,  maintainers 
were  interviewed  to  help  identify  possible  impacts  relating  to  Doctrine,  Organization, 
Training,  Material,  Leadership,  People  and  Facilities  (DOTMLPF). 

The  JITC  analysts  then  validated  the  DOTMLPF  impacts  against  the  ARGCS 
Concept  of  Operations.  DOTMLPF  impacts  were  updated  throughout  the  course  of  the 
ACTD  and  then  be  incorporated  into  the  comprehensive  JMUA  Assessment  Report 
released  in  November  2007. 32 


30  “Agile  Rapid  Global  Combat  Support  Advanced  Concept  Technology  Demonstration  Integrated 
Assessment  Plan”  Joint  Interoperability  Test  Command,  FT  Huachuca,  AZ,  May  2006. 

31  Ibid.. 

32  Ibid. 
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F.  EQUIPMENT  CONTROLS 

In  conducting  JMUA  assessments  at  selected  service  facilities,  it  was  necessary  to 
use  local/legacy  ATS  equipment  as  well  as  current  weapons  systems.  In  the  case  of  the 
Navy  this  was  a  standard  CASS  bench  and  a  unit  level  F/A-18C.  Existing  equipment  is 
selected  through  operational  units  within  an  organization  that  have  already  agreed  to 
participate  in  the  assessment.  In  similar  fashion  to  the  selection  of  participating 
maintainers,  organizations  (squadrons)  are  chosen  based  incumbent  leadership  interest. 
The  JITC  believes  this  method  results  in  a  more  realistic  test  and  reduces  the  number  of 
ambiguities  in  the  final  report.33 

G.  DATA  PROCESSING  AND  ANALYSIS 

Preliminary  analysis  of  the  JMUA  data  is  conducted  in  the  field  as  a  further 
verification  of  data  quality.  Anomalies  encountered  help  identify  data  entry  errors. 
Exhaustive  analysis  of  the  data  is  then  being  completed  at  the  JITC.34 

At  the  conclusion  of  each  demonstration,  JITC  conducts  an  on-site  briefing  on 
general  impressions.  A  JITC  final  report  is  due  45  working  days  after  completion  of  last 
assessment  briefing.35 

H.  EXAMPLE  RESULTS  TABLES36 

The  JITC  records  all  test  results  and  summarizes  them  in  tabular  form.  Because 
of  the  detailed  nature  of  the  information,  a  table  is  placed  in  an  appendix  to  the 
assessment  report  for  reference.  Table  2  provides  an  example  of  how  the  data  is 
displayed. 


33  Agile  Rapid  Global  Combat  Support  Advanced  Concept  Technology  Demonstration  IAP. 

34  Ibid. 

35  Ibid. 

36  Ibid. 
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Table  2.  Measure  of  Performance  Summary  Matrix  (Example) 


Critical  Operational  Issue/ 

Measure  of  Effectiveness/ 

Measure  of  Performance 

Threshold 

Summary  Results 

Remarks 

LMUA 

JMUA 

Total 

CRITICAL  OPERATIONAL  ISSUE  (COI)  1  Does  ARGCS  provide  interoperability  between  Services’  support  equipment  (and 

limited  interoperability  between  each  Service’s  Organizational,  Intermediate  and  Depot  maintenance  levels)?  (Effectiveness) 

MOE  1.1:  The  ARGCS  is  compatible  with  legacy  system  test  program  sets. 

1.1.1  Consolidated  Automated 

Support  System  (CASS). 

Percentage  of  selected 

CASS  test  program  sets  that 

are  ARGCS  compatible. 

85% 

100% 

(7/7) 

100% 

(1/1) 

100% 

(8/8) 

1.1.2  Integrated  Family  of  Test 

Equipment  (IFTE). 

Percentage  of  selected  IFTE 

test  program  sets  that  are 

ARGCS  compatible. 

85% 

80% 

80% 

80% 

Two  TPSs  not  fully  compatible. 

Legend: 

ARGCS-  Agile  Rapid  Global  Combat  Support 

CASS-Consolidated  Automated  Support  System 

IFTE-Integrated  Family  of  Test  Equipment 

JMUA-Joint  Military  Utility  Assessment 

LMUA-Limited  Military  Utility  Assessment 

TETS-Third  Echelon  Test  Set 

The  JITC  assigns  a  subjective  overall  rating  to  each  MOP  based  on  a  combination 
of  test,  survey,  and  interview  results  and  direct  observation.  This  infonnation  is 
displayed  in  the  main  body  of  the  assessment  report.  Green,  Yellow,  and  Red  color  codes 
signify  the  rating  given.  The  following  definitions  define  the  ratings  and  Table  3 
exemplifies  the  manner  in  which  the  ratings  are  displayed:37 

•  Green:  Provides  substantial  utility.  Minor  improvements  may  be  needed. 
Additional  data  may  be  required  to  supplement  some  findings,  but  do  not 
affect  the  overall  assessment. 


37  Agile  Rapid  Global  Combat  Support  Advanced  Concept  Technology  Demonstration  Integrated 
Assessment  Plan”  Joint  Interoperability  Test  Command,  FT  Fluachuca,  AZ,  May  2006. 


22 


•  Yellow:  Provides  utility.  Improvements  may  be  needed.  Additional  data 
may  be  required  in  some  areas  to  support  a  Green  assessment. 

•  Red:  Provides  little  or  no  utility.  Major  improvements  required. 


Table  3.  MOE/MOS  Results  (Example) 


(i)  MOE/MOS 

LMllA 

JMUA 

Overall  j 

MOE  1.1  The  ARGCS  is  compatible  with  legacy 

system  test  program  sets. 

Green 

Green 

Green 

MOE  1.2  The  ARGCS  has  the  capability  to  share  diagnostic 

information  between  Services  and  a  Service’s  maintenance 

echelons  (Levels). 

Amber 

Amber 

Amber 

MOE  1.3  The  ARGCS  is  interoperable  with  components  and 

standard  tools  within  common  operating  environments. 

Green 

Green 

Green 

MOS  2.1  The  ARGCS  provides  accelerated  support  equipment 

deployment. 

Amber 

Green 

Green 

MOS  3.1  The  ARGCS  effectively  supports  diagnostic  work. 

Green 

Green 

Green 

MOS  4. 1  The  ARGCS  will  drive  lower  lifecycle  costs. 

Green 

Green 

Green 

MOS  5.1  The  ARGCS  will  reduce  the  logistical 

footprint. 

Green 

Green 

Green 

MOE  6.1  The  ARGCS  effectively  supports  diagnostic 

work. 

Amber 

Green 

Green 

MOS  7.1  The  ARGCS  is  usable  by  typical  operators  in 

the  operational  environment. 

Green 

Green 

Green 

Legend: 

ARGCS-  Agile  Rapid  Global  Combat  Support 
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I.  SUMMARY  OF  CONTROL  SYSTEMS 

The  IAP  states  that  the  purpose  of  the  assessment  is  to  “assess  the  ARGCS  system 
to  determine  the  extent  to  which  it  adds  military  utility  to  the  Services  maintenance 
operations,  methods,  or  procedures.”38  Based  on  a  review  of  the  plan  itself,  the  goals  of 
management  and  the  controls  included  within,  it  appears  JITC  planned  well  to  this 
mission. 

Control  systems  in  the  case  of  a  test  and  evaluation  program  are  put  in  place  to  aid 
in  the  acquisition  of  accurate  and  usable  data.  In  the  case  of  ARGCS,  a  very  specific  and 
detailed  IAP  was  authored  based  on  successful  testing  strategies  identified  by  the  Joint 
Forces  Command,  including  an  overall  management  plan.  As  control  systems,  this 
direction  is  put  in  place  to  guide  the  behavior  of  individuals  responsible  for  conducting 
the  testing  and  the  analysis  of  the  data  gathered.  The  ARGCS  IAP  thoroughly  ensures 
that  testers  and  evaluators  understand  what  is  expected  of  them,  and  that  they  are  aware 
the  goal  of  an  accurate  and  honest  assessment  of  the  technologies.  In  this  case, 
management  control  (JITC  and  USPACOM)  have  taken  the  necessary  steps  to  ensure  that 
the  testers  and  evaluators  are  doing  what  is  best  for  DoD.39 

The  overall  demonstration  methodology  develops  and  excellent  test  strategy  that 
not  only  provides  an  unbiased  assessment  of  the  ARGCS  technologies  but  will  also 
appropriately  shares  the  project  risk  among  all  the  shareholders  (DoD,  USPACOM,  JITC 
and  Northrop  Grumman). 


38  “Agile  Rapid  Global  Combat  Support  Advanced  Concept  Technology  Demonstration  Integrated 
Assessment  Plan”  Joint  Interoperability  Test  Command,  FT  Huachuca,  AZ,  May  2006. 

39  Merchant,  Kenneth  A.,  Van  der  Stede,  Wim  A.,  “Management  Control  Systems:  Performance 
Measurement,  Evaluation  and  Incentives,”  Prentice  Hall  2003. 
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V.  ARGCS  ECONOMIC  ANALYSIS 


A.  INTRODUCTION 

The  goal  of  the  ARGCS  ACTD  is  to  use  new  proven  technologies  such  as 
synthetic  instrumentation,  open  system  architecture  and  an  advanced  dual  core  processor 
to  integrate  automatic  test  system  (ATS)  hardware  and  software  with  emergent 
technologies  such  as  the  RMOS  and  a  net-centric  support  system  to  provide  improved 
electronic  systems  maintenance  to  that  currently  proved  by  CASS  with  a  reduced  test 
system  footprint.40 

Specifically,  ARGCS  open  system  architecture  makes  it  a  flexible  system  that  can 
be  easily  reconfigured  with  advanced  technologies  and  be  used  as  another  Service’s 
tester.  The  net-centric  capability  is  focused  on  three  key  areas,  overall  network 
infrastructure,  common  processes,  and  data  management.  The  network  infrastructure  is 
designed  to  allow  an  maintainers  technical  access  to  the  most  up  to  date 
diagnostic/maintenance  data.  The  common  processes  involved  are  focused  on  providing 
a  systematic  approach  effectively  managing  data  with  ultimate  goal  of  quicker,  more 
accurate,  effective  diagnostics.41 

If  paired  with  an  accessible  network  the  entire  ADSR  system  will  enable  the 
reach-back  capability  that  would  allow  maintainers  at  the  organizational  level  to  capture 
Weapon  Systems  Built-in-test  (BIT)  and  other  test  data,  as  well  as  interface  with  data  at 
the  Intermediate  and  Depot  level  maintenance  echelons.  If  successful  this  diagnostic 
reasoning  capability  will  aid  the  maintainer  in  determining  which  WRA  to  replace  in  the 
case  of  ambiguous  test  results.  This  capability  can  significantly  reduce  false  pulls  and 
therefore  overall  maintenance  costs.  An  additional  benefit  is  the  ability  to  include  SMEs 


40  “Agile  Rapid  Global  Combat  Support  Advanced  Concept  Technology  Demonstration  Integrated 
Assessment  Plan”  Joint  Interoperability  Test  Command,  FT  Huachuca,  AZ,  May  2006. 

41  Ibid. 
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at  the  higher  echelons  during  the  diagnostic  phase  providing  direct  training  and  an 
increased  learning  curve  to  the  inexperienced  maintainer.42 

ARGCS  incorporation  of  proven  state-of-the-art  technology  has  resulted  in  the 
ability  to  significantly  reduce  the  physical  size  and  logistics  footprint  compared  the 
existing  CASS  equipment.  Additionally,  it  supports  access  to  electronic  and  technical 
orders  for  maintainers  further  reducing  the  overall  system  footprint  by  eliminating  the 
need  for  paper  manuals,  all  resulting  in  less  test  equipment  to  maintain/sustain  during 
deployment/employment  operations. 

The  synthetic  instrumentation  within  ARGCS  will  allow  maintainers  to  quickly 
update  the  system  when  new  capabilities  are  developed,  providing  the  capability  to  keep 
pace  with  increasing  test  requirements  posed  by  newer  weapons  systems.  It  is  a  proven 
technology  that  is  a  collection  of  hardware  and  software  modules  that  function  in  the 
same  manner  as  a  standard  test  instrument,  reducing  hardware  requirements  and 
facilitating  the  introduction  of  new  test  capabilities  via  software  modification  vice  the 
replacement  of  hardware. 

B.  SYSTEM  OBJECTIVES 

The  main  objective  for  ARGCS  is  to  address  a  set  of  significant  capability  gaps: 
lack  of  interoperability  of  support  equipment;  delays  in  fielding  capable  support 
equipment  for  newly  fielded  systems,  difficulties  and  delays  in  support  equipment 
upgrades,  high  support  costs;  large  logistic  footprints,  and  high  false  failure  detection 
rates.43 

ARGCS  technologies  are  intended  to  provide  a  set  of  improved  operational 
capabilities  that  directly  address  the  identified  capability  gaps  and  provide  benefits  in  the 
following  areas.44 

•  More  capable  and  productive  test  system  equipment: 

42  Agile  Rapid  Global  Combat  Support  Advanced  Concept  Technology  Demonstration  IAP. 

43  This  list  is  based  on  the  Briefing  on  Agile  Rapid  Global  Combat  Support  (ARGCS)  Business  Case 
Analysis,  April  2006. 

44  Ibid. 
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o  Resulting  from  quicker,  more  accurate  fault  diagnosis  -  due  to 

■  Higher  quality  data  recorded  and  shared  at  the  various  levels 
(Organizational,  Intermediate  and  Depot); 

■  Net-enabled  diagnostics  based  on  analysis  of  relevant  maintenance 
experience  and  near-real-time  access  to  subject  matter  experts; 

■  Less  time  to  diagnose  faults; 

o  Higher  test  system  reliability  and  availability  due  to  self-test  features, 
reduced  calibration  times,  and  uninterruptible  power  supplies; 

o  Leading  to  leading  to  fewer  test  systems  required  and  a  corresponding 
reduction  in  tester  footprint; 

o  Leading  to  better  transportability  for  deployed  operations  due  to  fewer  test 
systems  needed; 

•  Fewer  ATS  types  in  service: 

o  Resulting  from  multiple-weapon  system  capabilities  in  one  type  of  test 
equipment  (commonality); 

o  Leading  to  reduced  logistics  support  costs  for  testers  across  services  and 
reduced  investment  costs  for  the  future  portfolio  of  ATSs; 

o  Enabling  joint  and  regionalized  maintenance; 

•  Fewer  obsolescence  problems: 

o  Due  to  system  scalability  and  open  architecture; 

o  Resulting  in  quicker,  cheaper  upgrades  and  faster  fielding  of  supporting 
automatic  test  equipment  for  new  combat  systems. 


C.  IDENTIFICATION  OF  ALTERNATIVES 

Those  writing  the  Request  for  Proposal  (RFP)  for  ECASS,  will  need  to  consider 
the  bundle  of  technologies  currently  referred  to  as  ARGCS.  Likewise,  the  RFP  will  need 
to  consider  the  advisability  of  including  eventual  provision  for  network-enabled 
maintenance  some  time  in  the  future.  The  alternative  is  to  extend  the  life  of  CASS  until 
the  networked  features  of  ARGCS  have  been  fully  evaluated.  A  second  comparison  will 
be  made  of  not  only  developing  ECASS  with  all  inherent  ARGCS  technologies  but  also 
concurrent  development  of  the  network  needed  to  support  the  net-centric  capabilities 
within  the  ARGCS  system  as  apposed  to  just  proceeding  forward  with  ECASS  as  a  stand 
alone  tester  and  delaying  the  development  of  the  network. 


27 


This  analysis  will  perform  a  comparison  of  alternatives  using  current  CASS 
maintenance  costs,  and  the  assumed  costs  of  ARGCS  enabled  ECASS  and  the  supporting 
network.  The  analysis  will  be  qualitative  in  its  discussion  of  the  proven  ARGCS 
technologies  and  quantitative  in  its  use  of  Return  on  Investment  (ROI)  calculations  using 
data  from  the  Franck  BCA  and  the  Bello  Model  to  analyze  cost  comparisons. 

D.  ASSUMPTIONS  AND  ESTIMATE  OF  COSTS 

Driving  the  need  for  a  replacement  ATS  for  the  Navy  are  the  desire  for  reduced 
test  run  times  and  overall  reduced  costs.  The  emergent  technologies  with  the  ARGCS 
ACTD  answer  the  need  for  reduced  test  run  time,  in  order  to  evaluated  the  benefits 
associated  with  the  ARGCS  ACTD  as  it  applies  to  reduced  costs  it  is  necessary  to  look  at 
the  operations  and  maintenance  (O&M)  costs  for  maintaining  the  current  CASS  ATS, 
compared  to  the  expected  O&M  costs  for  maintaining  ARGCS  enabled  ECASS  plus  the 
costs  associated  with  developing  and  producing  ECASS.  Then  working  in  the  proposed 
savings  associated  with  the  expected  increased  capabilities  of  ECASS  using  the  ARGCS 
technologies  to  reduce  false  pulls  and  as  a  result  increase  F/A-18C  operational 
availability. 

The  costs  associated  with  CASS  (including  the  expected  annual  increase  in  O&M 
costs)  and  ECASS,  that  will  be  used  in  the  comparisons  to  follow  were  obtained  from 
NAVAIR  PMA260,  they  were  provided  by  Mr.  Anthony  Geneva,  the  technical  manager 
for  the  evaluation  of  the  ARGCS  ACTD  and  Mr.  Bill  Ross,  the  NAVAIR  Deputy 
Director  for  Aviation  Technology  Development  (NAVAIR  PMA-260D).  Previous 
research  (Franck  BCA)  has  estimated  the  concurrent  development  of  the  net-centric 
capabilities  for  ARGCS  would  add  approximately  $400M  to  the  developmental  costs  of 
the  system.  Discussions  with  the  previously  referenced  experts  in  the  field45  and,  based 
on  CASS  Total  Ownership  Cost  experience,46  validate  this  as  a  realistic  figure. 


45  Source  is  Dr.  Daniel  Nussbaum,  Professor  NPS. 

46  Source  is  CASS  NAIRTOC  report  of  26  June  2006. 
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The  goal  in  making  the  following  to  provide  ARGCS  assessment  team  with  an 
independent  business  case  assessment  of  the  ACTD’s  proposed  benefits.  Due  to  the  fact 
that  ECASS  will  be  a  replacement  program  for  the  Navy  and  not  one  proposed  to  fill  a 
gap  in  “warfighter  needs”,  it  is  necessary  to  compare  expected  capabilities  and  cost  to  the 
best  available  capabilities  and  costs  associated  with  the  existing  system,  CASS. 

Although  the  RMOS  assessment  at  NAS  Lemoore  was  not  conclusive,  discussion 
with  experts  in  NAVAIR  indicates  that  existing  RMOSs  have  been  demonstrated  to  work 
and  therefore  the  assumption  that  the  RMOS  will  work  as  expected  will  be  part  of  the 
foundation  for  our  final  conclusions.47 

Although  problems  currently  exist  in  using  the  existing  networks  in  the  Navy 
Marine  Corps  Internet  (NMCI)  and  the  Global  Combat  Support  System  (GCSS)  to  enable 
the  reach  back  capability  proposed  as  part  of  the  net-centric  technologies  in  ARGCS,  it 
will  be  assumed  that  such  networks  will  be  available  with  the  expected  investment  cost  of 
$400M,  when  comparing  that  option. 

As  discussed  the  ECASS  program  is  intended  to  replace  the  CASS,  whose 
embedded  technology  is  several  generations  old.  Current  information  on  CASS  indicates 
that  it  will  rapidly  become  unsupportable  by  the  Commercial  Off  the  Shelf  (COTS) 
supply  chain  used  to  maintain  it.  As  a  result,  although  the  networking  goal  for  ARGCS  is 
a  valuable  goal,  the  main  goal  for  ECASS  is  to  be  the  next  generation  tester  not  a  new 
network  program.  It  has  been  stated  that  “ECASS  would  not  have  even  left  the  starting 
gate  if  the  requirement  was  to  develop  a  new  maintenance  network  rather  than  a  new 
tester,”  so  there  is  a  need  to  develop  a  separate  program  for  the  network  elements.48 

•  Cost  to  produce  ECASS/unit  $2.0M 

•  Cost  to  develop  ECASS  $60. OM 

•  Total  ECASS/ARGCS  units  to  be  fielded  439 

•  Annual  CASS  O&M  costs  $53K  *  1.10  /  year 

(10%  annual  increase) 

47  Email,  Mr.  Richard  Danz,  November,  2007,  NAVAIR  PMA  260. 

48Ibid. 
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Annual  ECASS/ARGCS  O&M  costs 
Est.  Annual  False  Pull  Savings  w/o  network 
Est.  Annual  OA  Savings  w/o  network 
Est.  Annual  False  Pull  Savings  w/  network 
Est.  Annual  OA  Savings  w/  network 
Concurrent  Development  of  the  Network 


$53K  *  .7  /  year 
$85M  (for  439  units) 
$65M  (for  439  units) 
$1 10M  (for  439  units) 
$90M  (for  439  units) 
$400M 


E.  IDENTIFICATION  OF  MISSION  RELATED  BENEFITS 

The  proposed  mission  related  benefits  for  the  introduction  of  ECASS  are:49 

•  Interoperability 

■  Re-configurable  to  support  multiple  weapons  systems 

•  Data  Sharing  Capability 

■  Net  centric  environment 

■  Most  up  to  date  data 

•  Reduced  logistical  footprint 

■  Smaller  unit  size 

■  Fewer  spares 

■  Less  unit  maintenance 

•  Quicker  and  more  accurate  fault  diagnosis 

■  Reduced  TPS  times 

■  Increased  accuracy  via  RMOS 

■  Reduced  false  pulls 

■  Reach  back  access  to  SMEs 

•  Faster  cheaper  capability  to  upgrade  tester  capability  (synthetic  instrumentation) 

■  Reduced  hardware  requirements 

•  Ease  of  system  upgrades 

•  Open  architecture  reduces  obsolescence  issue  in  the  future 


F.  ANALYSIS  DISCUSSION  AND  SUMMARY 

The  ARGCS  management  plan  listed  seven  Critical  Operational  Issues  (COI)  that 
relate  assessment  of  ARGCS  to  the  JWPs  listed  earlier,  within  those  COI’s  measures  of 


49 


“Agile  Rapid  Global  Combat  Support  Advanced  Concept  Technology  Demonstration  IAP. 
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effectiveness  (MOE)  were  established,  listed  below  those  MOEs  will  provide  the 
framework  for  the  discussion  on  the  analysis  of  alternatives.50 

COI 1:  Interservice  interoperability 

MOE  1.1:  The  ARGCS  is  compatible  with  legacy  system  TPS 
MOE  1.2:  The  ARGCS  has  the  capability  to  share  diagnostic 
information  between  Navy  maintenance  echelons. 

MOE  1.3:  The  ARGCS  is  interoperable  with  components  and 
standard  tools  within  common  operating  environments. 

COI  2:  Faster  accommodation  of  new  war- fighting  capabilities 

MOE  2.1:  The  ARGCS  provides  accelerated  support  equipment 
deployment. 

COI  3:  Rapid  upgrades  to  support  tomorrow’s  weapon  system  perfonnance 
requirements. 

MOE  3.1:  The  ARGCS  technology  provides  the  capability  to 
rapidly  upgrade  the  ATS. 

COI  4:  Reduction  in  support  costs  for  maintaining  current  weapon  systems 
MOE  4. 1 :  The  ARGCS  will  drive  lower  life  cycle  costs. 

COI  5:  Reduction  in  logistical  footprints  (both  weight  and  volume). 

MOE  5.1:  The  ARGCS  will  reduce  the  logistical  footprint. 

COI  6:  Reduction  in  false  pulls 

MOE  6.1:  The  ARGCS  effectively  supports  diagnostic  work.  And 
reduces  unnecessary  repair  actions  (false  pulls) 

COI  7:  Usable  by  intended  users  in  its  operational  environment 

MOE  7.1:  The  ARGCS  is  usable  by  typical  operators  in  the 
operational  environment. 

MOE  7.2:  The  ARGCS  effectively  performs  self-diagnostics. 

The  MOEs  provide  an  excellent  framework  for  this  discussion.  As  discussed 
earlier  the  new  technology  within  the  ARGCS  ACTD  (dual  core  processor,  open 
architecture,  synthetic  instrumentation)  are  all  proven  technologies  and  performed  as 
such  during  the  JMUA  assessment  in  Lemoore.51  The  only  portions  of  the  MOEs  for  the 
ARGCS  ACTD  that  were  not  completely  validated  were  those  that  are  directly  effected 
by  the  RMOS;  lower  lifecycle  costs,  smaller  logistical  footprint  and  better  diagnostics 
(fewer  false  pulls).  The  successful  performance  of  these  technologies  and  the  escalating 


50  “Agile  Rapid  Global  Combat  Support  Advanced  Concept  Technology  Demonstration  Integrated 
Assessment  Plan”  Joint  Interoperability  Test  Command,  FT  Huachuca,  AZ,  May  2006. 

51  Ibid. 
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costs  to  maintain  CASS  appear  to  be  reason  enough,  on  a  qualitative  basis,  to  move 
forward  with  the  immediate  development  of  EC  ASS. 

If  we  only  look  at  maintenance  and  support  costs  for  both  systems  as  a  basis  for 
comparison,  the  quantitative  argument  is  not  as  strong  for  the  immediate  development  of 
ECASS.  The  Return  on  Investment  (ROI)  analysis  (Figure  1)  indicates  an  Internal  Rate  of 
Return  (IRR)  of  only  1  percent  when  considering  only  the  O&M  savings  promised  by 
ECASS.  The  goal  of  this  comparison  was  to  only  look  at  the  savings  associated  with  new 
equipment.  In  this  case  the  comparison  is  made  by  comparing  the  escalating  costs  of 
maintaining  a  twenty  year  old  piece  of  ATE  to  development  and  purchase  costs  of  a  new 
piece  of  ATE  and  its  expected  reduction  in  maintenance  costs.  This  figure  is  based  solely 
on  the  purchase  and  O&M  costs  of  ECASS  developed  immediately  compared  to  delaying 
the  development  for  two  years  and  the  extended  O&M  costs  of  CASS. 

A  second  ROI  computation  (Figure  2)  using  these  O&M  savings  in  conjunction 
with  the  estimated  savings  due  to  decreased  false  pulls  (when  the  RMOS  functions 
effectively)  of  $85M  and  the  estimated  (Franck  BCA)  of  $65M  due  to  increases  in 
aircraft  operational  availability,  shows  an  increase  in  the  IRR  to  2 1  percent,  presenting  a 
much  more  robust  quantitative  case  for  the  immediate  development  of  the  ECASS  with 
the  ARGCS  technologies. 

The  third  analysis  assumes  the  immediate  development  of  ECASS  with  the 

inclusion  of  all  ARGCS  technologies  plus  the  immediate  development  of  the  support 

network  enabling  reachback  capability  and  database  updates.  This  analysis  compares  the 

immediate  development  of  the  network  with  waiting  five  years  with  the  hope  of  reducing 

costs.  There  is  reasonably  sufficient  evidence  to  support  the  $400M  price  tag  (Franck 

BCA)  placed  the  concurrent  development  of  the  associated  network,  as  well  as  a  potential 

50  percent  reduction  in  those  costs  if  the  development  were  delayed.  In  order  to  make 

this  a  realistic  comparison  a  20  year  period  was  considered.  Although  in  many  cases 

delaying  the  acquisition  of  a  new  system  tends  to  increase  overall  acquisition  costs,  in 

this  case  it  can  be  expected  to  reduce  them  due  to  the  fact  that  a  supporting  network  isn’t 

currently  available  but  is  in  development  to  support  other  joint  net-centric  programs.  It  is 

very  possible  that  delaying  the  specific  development  of  a  network  to  support  ECASS,  can 
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significantly  reduce  the  amount  of  money  the  Navy  will  spend  on  the  ECASS  program  if 
it  desires  the  network  to  be  available  when  ECASS  is  deployed.  Figure  3  shows  that  if 
waiting  five  years  to  develop  full-up  network  capability  cuts  development  costs  by  50 
percent,  then  there's  a  good  case  for  waiting,  immediate  development  has  an  IRR  of  only 
2.7  percent.  A  sensitivity  analysis  of  the  results  is  shows  that  if  cost  were  reduced  by 
only  a  quarter  the  IRR  would  jump  to  7.4  percent  and  to  10  percent  if  the  savings  were 
only  10  percent. 
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Figure  1.  ROI  of  Retaining  CASS  indefinitely  vs.  immediate  development  of  ECASS 
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Figure  2.  ROI  of  delaying  ECASS  development  for  two  years  vs.  immediate  development 
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Figure  3.  Wait  five  years  to  develop  full  network  capabilities  to  reduce  costs. 
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VI.  CONCLUSION  AND  RECOMMENDATION 


Current  projections  indicate  that  the  O&M  costs  for  CASS  will  continue  to  grow 
in  the  near  future  (minimum  10  percent/year),  but  the  business  case  for  moving  forward 
quickly  with  ECASS  incorporating  ARGCS  technologies  (based  solely  on  O&M  costs)  is 
not  particularly  robust.,  However,  if  estimated  savings  gained  from  increased  F/A-18C 
operational  availability  and  estimated  savings  from  fewer  false  pulls  are  also  included, 
the  IRR  jumps  to  21  percent  for  the  standalone  ARGCS-enabled  ECASS  unit.  Currently 
in  the  commercial  sector  an  ROI  evaluation  that  yields  a  return  greater  than  7  percent  is 
considered  significant,  however  in  this  case  since  the  primary  goal  for  ECASS  is  to  be  a 
smaller  less  costly  replacement  for  CASS  with  the  capability  to  be  easily  upgraded,  a 
goal  which  previous  discussion  shows  that  it  meets,  any  IRR  that  is  not  a  cost  (a 
negative)  is  to  be  considered  to  have  met  the  analysis  “hurdle  percent”. 

Additionally,  even  if  we  avoid  any  speculation  and  strictly  focus  on  the  proven 
technology  within  the  ARGCS  ACTD,  that  would  be  a  significant  portion  of  the  ECASS 
tester,  the  case  to  move  forward  is  quite  strong.  The  significant  portions  of  the  ACTD, 
such  as  the  dual-core  processor,  open  system  architecture  and  synthetic  instrumentation 
all  work.  The  JMUA  in  Lemoore  proved  that  the  ARGCS  ACTD  works  as  well  as  the 
current  state  CASS  as  a  tester  and  that  the  piece  that  kept  the  ACTD  from  performing 
diagnostics  faster  was  the  existing  TPS.  If  the  decision  to  proceed  with  ECASS/ARGCS 
is  made,  TPS  hardware  and  software  will  be  developed  to  augment  the  added  speed  and 
capability  included  in  the  unit. 

A.  SUMMARY  OF  BENEFITS 
1.  Proven 

■  Interoperability  through  the  use  of  Open  System  Architecture 

■  Reduced  logistic  footprint 

■  Quick  and  accurate  fault  diagnosis 

■  Synthetic  Instrumentation  Capability 
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2.  Expected 

■  Net-Centric  Diagnostics  and  better  diagnostics  will  lead  to  less  false  pull  rates 
and  a  reduction  in  the  number  of  weapon  system  spares  needed 

NAVAIR  has  indicated  that  RMOSs  have  worked  elsewhere  and  that  any  help 
from  an  operational  reasoner  will  be  a  significant  upgrade,  therefore  not  proceeding  due 
to  the  inconclusive  results  of  the  RMOS  performance  at  the  JMUA  would  be  not  be 
advisable. 

Additionally,  the  Navy  should  recognize  that  a  strong  case  can  be  made  for 
including  provisions  for  future  network  capability  when  developing  ECASS.  Although  a 
strong  case  can  be  made  for  developing  the  network  now,  if  significant  savings  can  be 
achieved  by  waiting,  and  still  having  the  network  available  when  ECASS  starts  Limited 
Rate  Initial  Production  (LRIP)  there  would  be  a  significant  increase  in  ROI.  Also,  it 
needs  to  be  noted  that  developing  the  network  was  not  part  of  the  ARGCS  IAP,  and  even 
if  the  network  development  is  delayed  indefinitely  ECASS  should  still  be  developed  with 
that  ability  in  order  to  make  full  use  of  the  network  capability  when  it  does  become 
available. 

Although  the  estimated  costs  for  producing  ECASS  are  likely  accurate, 
acquisition  programs  have  a  history  of  going  over  budget;  if  there  are  program  budget 
cuts  are  for  this  reason,  program  managers  should  be  advised  not  to  make  those  cuts  in 
the  net-centric  features  of  ECASS.  Although  ECASS  will  perform  well  as  a  tester,  a 
significant  capability  leading  to  significant  future  savings  could  well  be  lost.  That  the 
ECASS  is  being  developed  primarily  as  a  standalone  test  system  should  not  be  reason  to 
ignore  the  advantages  of  network  capability.  Doing  so  would  be  analogous  to  building  a 
computer  without  an  internet  hookup,  just  because  the  internet  is  not  yet  fully 
operational. 

The  Bello  Discrete-Event  model  and  the  Franck  BCA  also  indicate  there  will  be 
significant  false  pulls  savings  with  a  standard  ECASS  and  operating  RMOS.  The  results 
of  the  simulation  indicate  that  the  reasoners  ability  to  resolve  ambiguous  tests  could 
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result  in  up  to  a  50  percent  reduction  in  false  pulls  and  30  percent  increase  in  aircraft 
operational  availability,  saving  over  $65M  and  $85M  respectively,  throughout  the  Naval 
Aviation. 

The  model  also  indicated  that  the  proposed  improved  capability  with  the  ARGCS 
system  to  reduce  TPS  run  times  by  15  percent  would  result  in  a  3  percent  reduction  in 
spares  required.  Since  an  O-level  organizations  will  always  carry  as  many  spares  as  it 
has  room  for,  the  savings  in  this  case  manifest  themselves  in  carrying  a  higher  percentage 
of  spares  for  the  WRAs  that  fail  more  frequently.  As  the  RMOS  builds  its  database,  in 
addition  to  helping  solve  ambiguous  test  results,  it  also  designates  the  percentage  of  time 
a  particular  WRA  fails  in  proportion  to  the  others  being  analyzed  in  a  particular  TPS. 
This  information  can  be  used  to  equip  maintenance  organizations  with  a  more  appropriate 
spare  parts  mix,  as  well  as  identify  those  WRAs  that  need  mean  time  to  failure 
improvements.  It  may  also  result  in  a  smaller  footprint  of  WRAs  at  all  levels  in  the 
future. 

Based  on  a  significant  IRR  and  potentially  better  operational  availability  and  false 
pull  rates,  significant  consideration  should  be  given  to  concurrent  development  of  the 
network  with  ECASS. 

In  assessing  the  current  state  of  CASS  and  its  escalating  O&M  costs,  the  positive 
IRR  of  replacing  CASS  with  ECASS  and  the  successful  performance  of  ARGCS  as  a 
WRA  tester,  it  appears  that  a  solid  business  case  exists  for  moving  forward  quickly  with 
development  of  ECASS. 
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